Proxy servers are intermediate servers that accept requests from clients and forward the requests to other proxy servers, a source server, or service the request from their own cache. Proxy servers are computing devices connected to a network that serve as intermediaries for customer computing devices, requesting services or data from “network resources”, accepting the requests from customers proxy servers and either forward the request to another proxy, redirect to the requested resource, or serve the request from their own cache, located locally or remotely, or distributed across multiple systems.
The proxy is also called ‘server’ or ‘gateway’. A proxy allows users on a network to browse the Web, send files over File Transfer Protocol (FTP), and work with e-mail and other Internet services. A proxy basically does the following: i) receives a request from a client inside the firewall; ii) sends the request to the remote server outside of the firewall; iii) gets the response; and iv) sends the response back to the client.
The proxies can be divided into different types depending on what functions are provided or what servers are used. The proxies can also be divided into Residential Internet Protocol (IP) proxies, Datacenter IP proxies, and Mobile IP proxies. A Residential IP proxy is an IP address linked to a physical device, for example, mobile phone or desktop computer. The real owners of the Residential IP proxies, namely, Internet service providers (ISPs), register residential IP addresses in public databases, which allows websites to determine a device's internet provider, network, and location. Datacenter IP proxy is the proxy server assigned with a datacenter IP. Datacenter IPs are IPs owned by companies not by individuals. The datacenter proxies are actually IP addresses that are not located in a natural person's home. Instead, the datacenter proxies are associated with a secondary corporation. Mobile IP proxies act similar to residential proxies. A mobile IP proxy is essentially one IP address that is obtained from mobile operators. Mobile IP proxies use mobile data, as opposed to a residential proxy that uses broadband ISPs or home Wi-Fi. A proxy server is basically a computer on the internet with its own IP address that the client's computer knows. When a client sends a web request, the request goes to the proxy server first. The proxy server then makes the web request on the client's behalf, collects the response from the web server, and forwards the web page data so that the client can see the page in the browser. When the proxy server forwards the web requests, the proxy server can make changes to the data but yet provide the data requested. A proxy server changes the client's IP address, so the web server is not provided the geographical location of the client. A proxy can encrypt the client's data so that the client's data becomes unreadable in transit. Also, a proxy server can block access to certain web pages, based on IP address or domain name.
Modern proxy servers do much more than simply forwarding web requests. For example, modern proxy servers can perform several additional functions to increase data security and network performance. Proxy servers act as a firewall and web filter, provide shared network connections, and cache data to speed up common requests. Proxy servers can provide a high level of privacy. Proxy servers change the IP address and other identifying information the web request contains (e.g., so that the destination server doesn't know who made the original request). Proxy servers can also be used to control internet usage of employees and children (e.g., organizations and parents set up proxy servers to control and monitor how their employees or kids use the Internet) or improve browsing speeds and save the bandwidth (e.g., proxy servers can cache (save a copy of the appropriate website data locally) popular websites and when a request is made will send the saved copy to the client).
Proxies can be used to bypass certain Internet restrictions (e.g. firewalls) by enabling a user to request the content through a (remote) proxy server instead of accessing the content directly. Proxy servers are often used to get around geo-IP based content restrictions. If someone wants to get content from, for example a US webpage, but they do not have access from their home country, they can make the request through a proxy server that is located in the USA (and has a USA IP address). Using proxy services, the user's traffic seems to be coming from the USA IP address. Proxies can also be used for web scraping, data mining, and other similar tasks.
A proxy provider can control the quality of proxies and decide which Internet Protocol (IP) addresses are going to be served by a set of proxies used by clients. If the same proxy is used for too many requests, the proxy will ultimately be banned by the Internet service provider or the web page and it will not be possible to use such proxy to make subsequent requests. If too many requests come in from one IP address in a short period of time, then the site will return some sort of error message and disallow the requests from that proxy for a pre-set period of time. To prevent errors or disallowed requests, the proxies are rotated according to usage rate or other predetermined factors. The rotation is quick enough so that the target website server does not provide any errors and does not prevent using a proxy for additional requests.
While some systems operate individual proxies, managing lists of proxies and helping select an individual proxy for a particular user or user device for a prolonged session, these systems do not provide for an abstraction layer of proxies comprising a logical entity of a proxy provider, and managing proxy service access on a larger scale.
While some systems detect the proxy availability based on one or several requests occurring during a short period of time, these systems do not keep statistical information about the use of the proxies. Also, these systems do not evaluate proxies during longer periods of time and do not summarize proxy service quality and availability. Further, current rotation methods do not change the overall configuration of the proxy utilization.
For example, some proxies are evaluated taking into account several requests and listing the proxies from the most available to the least available. The process might be continuously repeated, but the overall statistics are not kept or measured. Also, according to changes in proxy availability and after the value of the proxy changes, the Web Browser needs to be periodically and dynamically updated with the “best” Proxy/Socks Server, so the proxy list needs to be updated constantly. Additionally, the prior art does not disclose the automatic collection of the statistical data and automatic configuration of thresholds without additional need for intervention. Also, the prior art systems require constant human supervision and intervention in summing up the generated data and changing the availability of proxies.
Further, the anonymity of the request or the privacy of the client making a request is not available. The traditional approach is to terminate the encrypted session within the domain of the proxy management provider, which compromises the privacy of the requests.